Phonic engine

ABSTRACT

A method and apparatus which allows a user to specify phonemes and the relative positions of the phonemes with respect a word or a group of words, such as at title. Phonemic indicators representing phonemes are provided, and selection of the indicators is allowed. When a phoneme is selected, its position with respect to a word or group of words is determined. According to one aspect of the invention, the phonemes are either initial phonemes or final phonemes, i.e., phonemes which are identifiable by young children. According to another aspect of the invention, the indicators are provided in a manner understandable to an individual who cannot read, such as a young child. According to another aspect of the invention, the phonemic indicators further indicate positional information, in which case both the phoneine and its position are selected when the indicator is selected. According to another aspect of the invention, the selection process itself forms associations between the phonemes and their respective positions. According to another aspect of the invention, the selection may be received using electronic input means, such as a computer selection device, a remote control device, or from a computer network. According to another aspect of the invention, the word or group of words represent information corresponding to dictionary entries, encyclopedia entries, reference material, educaltional topics, database entries, games, merchandise or television programs.

This application is a division of U.S. patent application Ser. No.09/136,651 filed Aug. 19, 1998, which is a division of U.S. patentapplication 08/485,164 filed Jun. 7, 1995 now U.S. Pat. No. 5,799,267,which is a continuation in part of U.S. patent application Ser. No.08/278,928 filed Jul. 22, 1994, abandoned.

FIELD OF THE INVENTION

The present invention relates to the field of educational toolsgenerally, and specifically to educational tools for individuals who donot yet know how to read.

BACKGROUND

Computer technology has become ubiquitous in our society. Computers areincreasingly used in the workplace, in the home and at schools. Manyapplications for computer technology exist which increase productivityby performing, with minimal intervention or direction, a number ofrepetitious mechanical tasks that are a necessary part of the morecomplex tasks which are performed every day in our society.

The present invention is embodied in a method and apparatus which allowsa user to specify phonemes and the positions of the phonemes withrespect a word or a group of words, such as a title. A plurality ofphonemic indicators are presented. When an indicator is selected, both aphoneme and its position relative to a word or group of words isdetermined, This method and apparatus may be incorporated in a furtherspecification/selection method and apparatus whereby children (and otherindividuals who cannot read) may convert their minimal phonologicalknowledge into written language, or make selections of items, such asentries in a dictionary, using this minimal knowledge.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows a computer video display executing a process according tothe present invention within a software program.

FIG. 2 shows the video display of FIG. 1, displaying animation in thedisplay area.

FIG. 3 is a flow chart diagram showing an exemplary method according tothe invention.

FIGS. 4A and 4B show selector cards that are used in a further exemplaryembodiment of the invention.

FIG. 5 is a diagram of an alternate computer video display executing aprocess according to the present invention within a software program.

INTRODUCTION

The present invention is embodied in a method and apparatus that enablesyoung, pre-reading children to use components of written, phonetic orphonemic representations of language for communication. It may beimplemented in software, as a mechanical device, as a dedicatedelectronic device, or as a combination of the three.

The apparatus (hereafter referred to as the "Phonic Engine") allowschildren (and other users) to convert their minimal phonologicalknowledge into written language or make selections of items representedby words using only minimal language components. The inventor hasdiscovered that the following two concepts can be effectivelypaired/combined to provide an unexpected result:

(1) phonetic/phonemic knowledge of initial and final sounds are amongthe first bits of language knowledge acquired; and

(2) Two bits of orthographic or phonological information about a word orgroup of words provides sufficient information to select a perusable setof entries containing that word or group of words, from a suitablelarger set of entries. That is, early pre-reading phonic knowledge orbasic orthographic knowledge may be mapped to criteria which are thenconverted into a word or group of words, such as a title.

For example, a 5000 entry children's software dictionary, encyclopedia,or other work may be used (hereafter referred to as "the database"). Itis assumed that there are approximately 44 phonemes included in thewords in the database (corresponding to the number of phonemes in theEnglish language). For purposes of estimation, it is further assumedthat initial and final phonemes are randomly distributed among the entryheadings. (A database having this characteristic could be builtintentionally.) For this example, specifying initial and final phonemesnarrows down the 5000 entries=(5000/44)/44˜2.58 (mean average). If theseentries are then presented via spoken words and/or pictures, a child caneasily select the one that he or she intends to select.

Hereafter, a sound is indicated by enclosing a letter, blend, prefix,suffix or other sound description between slashes (/). For example, thephonemes represented by /n/ and /m/ in initial and final positions,respectively, appear in the words "name", "nighttime", "numb", andpossibly several other words from a children's dictionary. If a userwishes to find one of these words in a software dictionary, andspecifies the phonemes /n/ and /m/, the Phonic Engine presents the userwith these three options. In a software encyclopedia, the phonemes /longa/ and /n/ would specify a group of entries including, for example,"Abraham Lincoln".

In an exemplary embodiment of the Phonic Engine, means are provided for(1) initially selecting the phonemes; (2) creating and displaying a listof words containing the phonemes in initial and final positions; and (3)selecting the desired entry. For a pre-literate or non-literate person,visual and/or auditory cues may be provided. It is contemplated thatother orthographic information, such as prefixes and/or suffixes may beprovided in the initial selection process. As used in this application"orthographic information" includes information that is more complexthan just initial letters. It comprises, but is not limited to prefixes,suffixes, blends and root words. The exemplary Phonic Engine includesinput means. The method of inputting selections depends on theindividual hardware configuration of the user's system. One of ordinaryskill in the art of software development could readily program acomputer or workstation to receive user inputs from a keyboard or from amenu using a pointing device. Depending on the hardware configuration ofthe computer, inputs may optionally be received by alternative means,such as across a network, from scanning a document, from voice,interactive television, or from a remote control device.

It is contemplated that the Phonic Engine may be incorporated into othertypes of software programs, or dedicated devices. For example, thePhonic Engine may be incorporated within a software program foraccessing data in a reference document, a dictionary, an encyclopedia,an almanac, etc. When the information is stored as data in a database,entries of interest may be retrieved by the Phonic Engine using languageas the search criteria, even though the user does not know how to reador write.

In addition, it is contemplated that the Phonic Engine may be used toimplement control functions, for example, to access an electronicprogram guide database in a television system in order to select aprogram for viewing.

If the Phonic Engine is incorporated into a writing program or wordprocessor, it assists a non-literate person in writing a story. If thePhonic Engine is used in conjunction with a mechanical sorting device,it can form the basis of a word game or other game.

DETAILED DESCRIPTION

The first exemplary embodiment of the Phonic Engine is implemented assoftware in a reference work, specifically, a Talking Dictionary, amultimedia dictionary for children. The exemplary Talking Dictionary maybe executed on a computer configured for multimedia presentation, suchas a Macintosh computer including a display, pointing device, CD-ROMdrive, sound generation, and text to speech conversion capability. Otherhardware configurations may also be used.

TALKING DICTIONARY

The Talking Dictionary contains word entries. Each entry is associatedwith some or all of the members (attributes) listed in Table 1. It iscontemplated that other attributes, such as (but not limited to) shape,color, size may also be incorporated into the entries.

                  TABLE 1                                                         ______________________________________                                        (1)      the spelled word (optional)                                          (2)      phonetic/phonemic representation of the word                         (3)      definition                                                           (4)      usage example                                                        (5)      sound effect                                                         (6)      pronunciation                                                        (7)      picture (may include the spelled word in the picture)                (8)      animation (optional)                                                 ______________________________________                                    

The Talking Dictionary offers a variety of activities and ways to lookup a desired word; these methods are listed in Table 2.

                  TABLE 2                                                         ______________________________________                                        (1)     by identifying initial/final sound                                    (2)       by spelling                                                         (3)       by identifying attribute (such as shape, color and size)            (4)       by selecting the beginning letter and then perusing                              a list of alternative selections;                                (5)       the user may try to sound out randomly generated or                              selected sets of words - verified by the computer;               (6)       the user may find the initial/final sounds (and                                  other phonological units) of randomly generated or                            selected sets of words - verified by the computer;               (7)       matching picture or animation to a word;                            (8)       Write Story; or                                                     (9)       Read Story.                                                         ______________________________________                                    

Of the methods listed in Table 2, method (1) for finding a word byinitial and final sounds is described in detail, below, with referenceto FIGS. 1-3. Method (1) may be incorporated into a method for writing astory, which may be implemented as a separate application. The methodsfor writing and reading a story are also described below.

FIG. 1 shows the video display 10 of a computer executing an exemplaryTalking Dictionary program in accordance with the invention. The TalkingDictionary shown in FIG. 1 includes six specified fields within display10. Each field 11-16 is first described with reference to FIG. 1. Theprogram flow is then described below with reference to FIG. 3.

Referring to FIG. 1, the first field is an alphabetic strip menu 11.Alphabetic strip menu 11 is used to select letters. Letters are selectedusing a pointing device (e.g., by pointing the cursor 18 of a mouse atone of the letters in alphabetic strip menu 11 and clicking the mouse).When a letter is selected, the initial allographs 12a-12c associatedwith that letter are displayed in an allograph area 12. The allograph isdisplayed as part of a word with corresponding picture or animation. Theuser selects the desired initial allograph (e.g., 12a in FIG. 1) bydragging and dropping the allograph 12a into the initial sound box 14.The user selects the desired final allograph by selecting a letterassociated with the final allograph from alphabetic strip menu 11,selecting a desired allograph from a displayed list of allographscorresponding to the selected letter, dragging the desired allographfrom allograph area 12 and dropping the allograph into the final soundbox 16.

A list 17 of words 17a-17g within the Talking Dictionary database thatstart with the initial sound of the word in the initial sound box 14 andend with the final sound of the word in the final sound box 16 aredisplayed within a display area 13. One of the words 17a-17g is selectedby dragging and dropping the word into the word entry box 15.Alternately, the initial and final allographs and the target word may beselected by simply pointing to the corresponding picture and indicatinga selection, for example by "clicking" a button on the mouse. Theprogram can determine, from the context of the selection process,whether the selected allograph is an initial sound or a final sound andwhether the selected word is the target word.

OPERATION

The execution flow of the Talking Dictionary program is now describedwith reference to FIG. 3. At step 20, when the "Initial/Final Soundslook-up activity" program is executed, alphabetic strip menu 11,allograph area 12, initial sound box 14, final sound box 16, word entrybox 15 and a display area 13 may be displayed. A voice provides spokeninstructions on use of this activity.

Alphabetic strip menu 11 appears across the top of the screen. The stripmenu 11 contains the alphabet, and provides a means for indicatingphonemes. The user is then prompted (for example, by a spoken message)to select an initial sound. This is accomplished by selecting a letter.

In one exemplary system, a user may be assisted in selecting a letterfrom the alphabetic strip menu by software which pronounces the names ofthe letters as the selection pointer is dragged across the alphabeticstrip. In this instance, pointing at a letter without clicking isinterpreted as indicating a possible selection.

At step 30 a first letter is selected. Users approximately kindergartenage or older typically are sufficiently familiar with the alphabet toselect a letter appropriate for the desired sound much of the time. Theexemplary embodiment also assists users that do not have sufficientknowledge of the alphabet to identify the first letter. Pre-kindergartenchildren may also explore with the talking dictionary, receivingreinforcing animations and language knowledge while simply playing. Whenany letter is selected, the allographs that appear in the initialposition of words in the dictionary and begin with that letter aredisplayed underneath strip menu 11, in allograph area 12. For example,when the letter "c" is selected, the allographs 12a-12c representing/hard c/ (carrot), /soft c/ (city) and /ch/ (child) are displayed inallograph area 12. Both the word and an associated picture or animationmay be displayed, with the initial allograph highlighted, as shown inFIG. 1.

The more engaging the picture or animation, the better. If a child hearsa pig snort and say "oink" when he or she clicks on "oi", that allographis more likely to be remembered. If a child hears an amusing argumentbetween the two pronunciations of "either", he or she is more likely toremember that "ei" can represent either /long e/ or /long i/. Rules oforthography may also be presented (such as i before e, etc.)

Referring again to step 30 of FIG. 3, once the allographs 12a-12c aredisplayed in allograph area 12, the user may be prompted by a spokenmessage to select one of the allographs. The user selects an allograph12a by, for example, pointing the cursor 18 of a mouse (not shown) onthe allograph. When an allograph 12a is selected, a voice pronounces theallograph, along with any necessary explanatory information.

When the letter c is selected, as shown in FIG. 1, pictures of a carrot,city and child are shown, along with associated allographs c, c, and ch(perhaps others) and words. If the carrot is selected, as shown in FIG.1, a voice may say, "This is the letter c. It can make the sound /k/ asin carrot." If allograph 12b for city is selected, a voice may state,"This is the letter c. It can make the sound /s/ as in city." Ifallograph 12c is selected, a voice may state, "The letters ch togethercan make the sound /ch/ as in child."

At this point, a user that is unfamiliar with the alphabet may realizethat none of the allographs displayed for the letter "c" represents thedesired phoneme. The user then selects a different letter from alphabetstrip menu 11, and continues to select allographs until the user finds aletter and allograph that correspond to the first sound of the desiredword. When the user finds an allograph corresponding to the desiredinitial sound of the word, he or she drags the allograph and drops it inthe initial sound box 14. The Talking Dictionary displays arepresentation of that allograph in the initial sound entry box andstores the phoneme associated with that allograph in a temporaryvariable.

It is emphasized that the phoneme itself is actually selected by theexemplary selection method, not the allograph. For example, selection ofthe allograph k in initial position, along with t in the final positionwould represent carrot and cat, just as c would. And choosing the hard cin carrot (along with final sound /s/), would bring up class and kissduring a search of the database, but not circus.

In this way, grapheme/phoneme correspondence is presented, but notforced, because any allographs representing the same initial/finalsounds produce the same results. Thus, the child is gently exposed tographeme/phoneme associations, and encouraged to use this informationfor reinforcing material, but is not punished for a spelling error.

Referring again to FIG. 3, at step 40, a nearly identical (to step 30)procedure is followed for selecting the final sound. A letter isselected from alphabetic strip menu 11. Final allographs withcorresponding pictures and words which are associated with the selectedletter are displayed in allograph area 12. As shown in FIG. 1, selectingthe letter "p" to indicate the final allograph may, for (example, resultin a picture of a ship (which ends with the phoneme /p/ ). The finalallograph and its associated information are selected, dragged anddropped into the final sound entry box 16. A representation of the finalphoneme associated with that allograph is then stored in a temporaryvariable. At step 50, once initial and final sound selections arecomplete, a search loop may be entered to find all matching entries inthe dictionary database. The initial and final phonemes are matchedagainst the entries in the dictionary. Each entry in the databasecontains a phonetically (or phonemically) coded representation, and theindicated initial and final phonemes are compared against the initialand final phonemes of each entry.

Table 3 lists pseudocode that may be used to perform a search of entrieshaving the desired initial and final sounds.

                  TABLE 3                                                         ______________________________________                                        100      for all dictionary entries                                           110        compare initial phoneme                                            120        if match                                                           130         compare final phoneme                                             140          if match                                                         150           save in buffer (save word, pronunciation,                                         special information, and pointer to;                                          entry/record locator information)                           160           else                                                            170            move on to next                                                180         else                                                              190          move on to next                                                  ______________________________________                                    

Reference is again made to FIG. 3. If, at step 60, no word is found bythe search, then at step 70, a spoken voice message tells the user thatno words were found by the search. Control passes to step 30, and theuser is prompted to select a new initial sound.

At step 60, if any words are found by the search, this procedureproduces a list of matched entries, which is then presented to the userto peruse at step 80.

At step 80, the list 17 of matched entries 17a-17g is presented to theuser. For example, if the initial and final phonemes /k/ and /p/ areselecied in steps 30 and 40, then at step 80, the list 17 may appear asshown in FIG. 1. Because the phoneme /k/ is selected, the list 17 mayinclude words that begin with the phoneme /k/, regardless of spelling;in the exemplary display of FIG. 1, words that begin with the letters"c" and "k" are displayed. The list 17 may be presented as a list ofprinted words, as shown in FIG. 1, as a set of pictures, representingthe words, or as a combination of printed words and pictures.

If there are many entries (e.g., if the initial sound is /s/, whichcontains many blends), a secondary indicator sound may optionally beused to narrow down the list. For example, the user may be asked toidentify a third sound that appears in the word between the initialsound and the final sound.

Referring again to step 80 of FIG. 3, once the list 17 of words isdisplayed, the words are highlighted and pronounced in succession.Optionally, the voice message may also include special information aboutwords: homonyms, double pronunciations, etc. At any time, the user mayselect a word 17a-17g, using the pointing device, or by pressing the"enter" button while a word is highlighted. When a word is selected, itmay be verified by voice, saying for example: "The word you selected is`<word>`. Is this the word you want?"

If the user indicates "yes", by activating a control button on thescreen (not shown) then, at step 90, the word is displayed in the wordentry box 15 and an animation may be displayed in the display area 13.

Alternatively, once a user has initiated a search for items matching theinitial and other sound, a matrix containing pictures and/or words willbe displayed. This matrix may use several pages depending on the numberof matching entries that are found. When the user rolls the mousepointer over a picture or word, indicating the item as a possibleselection, if there is a word associated with the item it will bepronounced. When the user selects a picture or word, the display areamay show a definition or other material about the word. In addition, afilmstrip icon may be displayed allowing the user to view a shortanimation about the word. In addition, the program may pronounce theselected word. In this implementation, the information is not providedin a fixed sequence but the user is allowed to focus only on informationof interest using, for example, a "point and click" selection technique.Synchronization between animation and sound will occur as an artifact ofusing Video for Windows®, or Quicktime video either for Windows® or forMacintosh®.

Once an entry is located, it is displayed as a combination of theattributes listed in Table 1, according to sequencing information foundin the associated database record. The user may repeat the display byclicking on the selected word.

At step 90 of FIG. 3, an entry may be presented by the TalkingDictionary in a variety of ways as explained in the followingparagraphs.

For example, as shown in FIG. 2, selecting the phonemes /n/ and /m/ asinitial and final sounds may result in list 17 including the word"nighttime". If "nighttime" is selected, an animation 19 depicting adaytime scene turning into nighttime may be displayed in display area13. In another example (not shown), if the word "tall" is displayed, apicture or an animation of a skyscraper may appear in a display area 13.

An exemplary animation for the word "tall" (not shown) may include thefollowing: A boy and his mother are standing in front of a skyscraper.The boy looks up at the skyscraper and exclaims, "Wow, mom, that's atall skyscraper!" On top of skyscraper is King Kong. He jumps off,screaming "yippee i o ky ay!" Then King Kong's parachute opens, and hesmiles and winks. The boy and his mom giggle. Beneath the animation, thewords "Wow, mom, that's a tall skyscraper!" are displayed. Each word orgroup of words is highlighted as the boy utters the word. The word"tall" is emphasized. For example, "tall" may be displayed in adifferent color, reverse video or italicized. The user may select any ofthe words in the sentence using a pointing device or keyboard. As eachword is selected, it is pronounced. When the word "tall" is selected,the animation is repeated. The optional animation may be especiallyeffective for conveying and reinforcing concepts or abstract ideas suchas "near" and "far" or "freedom".

It will be understood by one of ordinary skill that instead ofanimation, a still picture of the skyscraper may be displayed in displayarea 13, and the sentence, "Wow, mom, that's a tall skyscraper!" may bedisplayed beneath the picture. Again, each word is highlighted as it isuttered by a voice. The user then has the same option of selecting anyof the words displayed.

It will be understood by one of ordinary skill in the art of designingeducational tools that some onomatopoeic words (or other words) may bereinforced by the production of sounds at step 90, without animation orpictures. The Talking Dictionary allows each word to be reinforced bythe respective medium/media that best suit(s) that word.

OTHER VARIATIONS

Many variations of the exemplary method and system are contemplated. Forexample, if the letter e is chosen at steps 30 or 40 of FIG. 3, severalpictures may be shown. The initial allographs associated with the letter"e" are displayed, along with associated pictures. A question mark and apackage tied with a bow may also be shown. Below the question mark isthe allograph "ei" and the word "either". When the question mark isselected, a voice responds, "The letters `ei` can make the sound /longe/ as in either. An animation demonstrating the concept of "either" isshown. When the package is selected, a voice responds "When you use theletters `ei` or `ie`, think of this: i before e except after c, or whensounding like /long a/ as in neighbor or weigh." An animation is shownto illustrate the point.

Optionally, when the question mark is selected, the voice may indicatethat "ei" can be pronounced /long i/ as in "either."

A variation is contemplated wherein the matched list is presented withgraphic indicators (pictures, animations, etc.) instead of, or inaddition to, the pronunciation.

Another variation is contemplated wherein the Phonic Engine is coupledwith an encyclopedia or other work. In this case, the text heading ofeach entry would be searched on, which may include more than one word.Also, the entry may be presented using a combination of one or moremedia elements, including pictures, movies, text, animation, or sound todepict an event, biography, place, thing, or other entry.

A variation is contemplated wherein the allographs are displayed withassociated animations instead of pictures. These may be randomassociated animations drawn from the Talking Dictionary's database.

Initial/final sound selection may also be accomplished via a number ofalternative methods which may be used instead of, or in conjunctionwith, the method shown in FIG. 3. One alternative is the use of asimplified phonetic/phonemic alphabet. Whereas the method discussedabove with reference to FIG. 3 allows selection of phoneme via grapheme,the alternative method would allow selection of phoneme directly.Optionally, this method may also provide secondarily an association withthe corresponding grapheme. Of course, the allographs presented in theselected process need not include all of the allographs in the graphemebut may include a subset, for example, only those allographs that arelikely to be familiar to a particular class of user or only thoseallographs which occur in the particular selection database that isbeing used.

Using the method of direct sound selection, the strip menu 11 wouldcontain a representation for /a/ /b/ /k/ etc., as well as /au/ /aw/ andothers. As an example of usage: if a user wishes to look up the word"back", he or she indicates /b/ as the initial sound, and /k/ as thefinal sound. When /b/ is selected, a picture of a boy appears along withthe letter "b". Then, a voice may say, "This is the sound /b/. It ismade with the letter `b`." If /k/ is selected, pictures of a kite and acarrot may appear, with the letters k and c, respectively. A voice maythen say: "This is the sound /k/. It can be made with the letters `k` or`c`."

Although the exemplary embodiment includes the use of an initial soundand a final sound, other embodiments are contemplated in which (1)initial and median sounds are used; and (2) initial, median and finalsounds are used. In another alternative variation, it is contemplatedthat various phonetic/phonemic combinations and/or orthographiccombinations may be used, based on or tailored to the knowledge of theuser. One may, for instance, incorporate prefixes or suffixes asselection criteria for a suitable audience.

If there is some phonetic/phonemic or orthographic knowledge which couldbe used as discrimination criteria by an individual who has a particularhandicap, that information may be incorporated to be used for leveragefor that individual.

For example, to suit a group of users who have difficulty discriminatinginitial sound (or for other reasons), final and other, median and other,or final letter and median sound, etc. may be used. To suit a group ofusers who have difficulty in sound discrimination (or for otherreasons), sounds and orthographic information may be remapped in anyfashion, to enable those users to use language via this method.

It is understood by one of ordinary skill in the art of computerprogramming that a variation of the exemplary method may be constructed,in which the user is given a choice between entering a median sound or afinal sound. The user may then use whichever combination of first soundand other (i.e., median or final) sound that provides a shorter list 17from which to select the desired word. It is also understood by one ofordinary skill in the art of computer programming that if list 17 islong, it may be presented in columns and rows or in any other suitablearrangement.

In another alternative variation, instead of having entries codedphonetically, a rule based algorithm may also be used for the search.

In a further alternative variation, a concordance structure representingall combinations of initial and final phonemes may be used. If aconcordance is used, each entry in this structure has pointers to (orindices of) words containing a respective combination of initial/finalphonemes in the dictionary database. The search would then consist oflocating the particular phoneme pair under consideration in thissecondary structure; retrieving index information: and then using thisinformation to retrieve the desired entry information from thedictionary database or another concordance.

It will be understood by one having ordinary skill in the art ofcomputer programming that the choice of grapheme/phoneme correspondence,or phoneme/grapheme correspondence may be implemented as an optionwithin the software. The two methods could be used together, forinstance augmenting a grapheme/phoneme selection list with aphoneme/grapheme list for vowels only.

Alternatively, three options (phoneme list, phoneme/grapheme list andgrapheme/phoneme list) may be provided, each option directed to users ofa respectively different stage in language development. A phoneme listmay be used first by users who have no experience with the alphabet. Ata later stage of language development, a phoneme/grapheme list is used.Eventually, the user's language skills become sufficiently advanced touse a grapheme/phoneme list.

It will be understood by one of ordinary skill in the art of programmingeducational software that the initial/final sound method of FIG. 3 maybe expanded into a method for writing a story, hereafter "Write Story".This variation allows a user to create an animated story, even if theuser is not able to read or write. A writing area (not shown) isdisplayed in which a story may be composed. The user may use one of theword look-up methods listed above in Table 2 to look up words that areadded (one at a time) to the story. Additionally, a mechanism isprovided whereby the user can write, using his or her own inventedspellings. If the user wishes to have an animation associated with theword, he or she may then have the word checked. If the word is notrecognized, the user is coached to verify the initial and final sounds,and this is used, in conjunction with other means, to help ascertain theword the user is trying to write.

If the database does not include the right word (e.g., if the usermisspells a word), the system may include a method for displaying ananimation for the misspelled word. The user may also select his or herown invented spellings along with animations.

Once the story is complete, it is displayed and/or read back to theuser. The user is then given the opportunity to select individual words;selecting one of the words initiates display of the associatedanimations and sounds.

The method hereafter referred to as "Read Story" provides a means tohave any written text spoken and displayed as an animated story. It alsoprovides a means for whole language instruction.

When the Read Story feature is selected, the user has an opportunity toselect a document. The document may be a letter, story, or otherdocument written by the user, a parent, friend, or teacher, or a wellknown piece of children's (or other) literature. The selected story isread aloud, and the individual words are highlighted as they are read.When a user selects a word, a corresponding animation is presented, fromthe internal database.

Because all or most of the words in the database have associatedpictures, animations, pronunciations, and/or sounds, this associatedinformation may be used to present a wealth of activities based on:

(1) picture/word correspondence;

(2) word/sound correspondence;

(3) word/letter correspondence;

(4) letter/sound correspondence, etc.

Amusing activities may also be included, such as mixing up definitionsand pictures, and letting children detect and correct the errors.

CREATING THE DATABASE

The number of words in the database (dictionary) is selected to besuitable for users in a desired age range or other desired criteria. Themaximum number of words that is displayed in list 17 (shown in FIG. 1)is related to the number of words in the database and the distributionof initial and final sounds for the entry headings in the database. Inan extensive database, or a database that, for some reason, has manyentries for a single initial/final phoneme pair, it is theoreticallypossible to overfill the display area (Once the database is complete,the maximum display, area needed to display text headings for eachinitial/final sound pair can be determined). In that case, a mechanismshould be provided for scrolling, paging, etc. Alternatively, the sizeof the dictionary and/or the distribution of initial and final sounds ofthe words in the dictionary may be adjusted to eliminate the chance ofoverfilling the display.

For each word, a data record is created containing the following:

1. the correctly spelled word (text) (optional)

2. phonetic/phonemic coding of the word (text) or numeric data

3. definition (text)

4. definition (digitized sound; if text to speech, the text definitionmay be used instead)

5. pronunciation (digitized sound; if using text to speech, thecorrectly spelled word may be used instead.)

6. sound effect (digitized sound)

7. picture

8. animation or still picture (may include word text as part of graphic)

9. usage example (text, if using text to speech; or digitized sound)

10. special information (exception information); codes representingvarious conditions, such as homonyms

11. sequence information

These records are grouped into a database. The database may be a table,arranged in alphabetical order by word, and stored on a sufficientlylarge medium. Because the entries are accessed by initial and finalsound, it may be desirable to create a secondary table of all thepossible initial/final sound combinations, to increase speed. Preferablythe table has pointers to the associated digitized graphic and soundentries. The table is stored in a medium that may be accessed rapidly bythe processor.

The animation may be stored as a PICS file or series of PICT files on aMacintosh computer manufactured by the Apple Corporation. The picturemay be stored as a PICT file on a Macintosh. Digitized sound may bestored as an AIFF file on a Macintosh. An authoring tool, such asMacromedia Director sold by Macromedia Corporation of San Francisco,Calif., may be used for creating parts of the Talking Dictionary.

The Talking Dictionary may be stored on CD-ROM or interactivetelevision. In this case, it may be preferable to utilize pointers toanimations and sounds, and to store them separately from the datarecords. This allows the data record to be cached in memory, or on thehard disk, providing more efficient access.

The special information fields may contain any special information thatmay inform the user of special attributes of the word. Examples ofspecial information include (but are not limited to), whether aparticular word/spelling, may have two pronunciations; synonyms,antonyms, homonyms, etc.

Sequence information is the order in which the word information isdisplayed. Some entries may have all fields present; others may not. Thevisual animation may be displayed with the definition, the usageexample, the sound effect, or other associated information. For example,with reference to the numbered attributes in Table 1, sequenceinformation 6, 3, 4/8 may indicate that the computer first pronouncesthe word; then states the definition; then displays the animation alongwith a usage example.

For the phonetic/phonemic coding of the word, the international phoneticalphabet may be used. Other systems may alternatively be selected, suchas are often found as front matter in printed dictionaries.

With respect to the multimedia elements, design trace offs are involvedbetween space, quality and the number of entries. Options include:storing sound and animation in compressed format, and using on-the-flydecompression for presentation of these elements. In a Macintoshenvironment, PackBits and UnpackBits OS calls may be used for thispurpose (these work best with pictorial information). Well knownalgorithms for lossless or lossy compression, for example LZW(Lempel-Ziv & Welsh), may be used. Suitable compression programs areavailable for the Macintosh from Aladdin Software, 165 Westridge Drive,Watsonville, Calif.

If necessary, space may be saved by: (1) providing animations in blackand white instead of color; (2) using a smaller bit depth; (3) usingdigitized speech at a low sampling rate; (4) using text-to-speechtechnology instead of, or in addition to, digitized speech. Othertechniques known to those of ordinary skill in the field of programmingapplications having graphics and sound may also be used.

Many alternative embodiments are contemplated. For example, the PhonicEngine may also be included in other reference works, in a writingprogram, as a command interpreter, or any other situation where a personwho does not know how to read or write wishes to understand or denotewritten language.

Some databases may have initial/final sound combinations (e.g., wordsbeginning with "s") for which a large number of entries are present. Oneof ordinary skill in the art of programming would readily understand howto add logic to obtain a secondary indicator. For example, the systemmay request that a user narrow down an initial "s" to include oreliminate a word beginning with an s blend. Alternatively, the systemmay ask for any median sound in the word.

The inventor is currently developing a prototype Talking Dictionary.This prototype deals with a specific subset of words: words depictinganimals. For purposes of this discussion, this shall be referred to asthe "Prototype." There are approximately 200 animals in this Prototype.

The exemplary Prototype is implemented using Microsoft Visual Basic,Professional Edition, Version 3.0 for Windows. The target environmentfor the prototype includes: Windows 3.1; CD-ROM (or other high capacityrandom access storage device); sound board; and speakers. Forperformance or other purposes, DLLs written in C or another compiledprogramming language may be utilized to accomplish certain tasks. Also,Windows API calls may be declared and used if necessary to enhanceperformance or achieve other objectives. As an alternative to VisualBasic, another programming environment such as C or C ++ may be used.

With reference to FIG. 1, the menu bar may be implemented using acommand button array; the allographs 12a-12c may utilize an imagecontrol array; the initial sound box and final sound box may also beimage controls; and the word box may be a picture control. It will beunderstood by one of ordinary skill in programming systems using GUI's,that alternate control types or custom controls may be used. Controlsfor displaying words may depend on whether the words are displayed astext or bitmapped graphics.

For presentation of sound (.WAV files) within the program, thesndPlaySound API function is used. Animations are displayed usingMicrosoft Video for Windows, compressed with the Microsoft RLE Codec.The MCIWNDX control, from the Microsoft Video for Windows developmentkit, is used for presenting the animations. The animations may berendered using Autodesk Animator Pro, along with other graphic tools.

A picture clip control may be useful in speeding upgraphic presentationof the allographs and associated pictures.

When the user initiates the Prototype, he or she views the menu bar, theinitial and final allograph boxes; and the word box. When the user rollsthe mouse over a letter, the letter is pronounced. When the user clickson a letter (using a pointing device) or otherwise selects a letter, theassociated allographs are displayed. The user preselects whether initialor final allographs are displayed by first clicking on the initial orfinal allograph box. When either of these is clicked, all allographs arecleared from the screen, and the user is presented with allographs ofthe type selected.

When the user rolls the mouse over a displayed allograph, informationabout the allograph is spoken, such as "This is the letter b. It makesthe sound /b/ as in balloon." If the user then clicks on this allograph,the allograph display (in this case a picture of a balloon, and the wordballoon) are displayed in the initial allograph box. Final allograph isselected in the same manner. If the user changes his or her mind, he orshe may replace the selected initial or final allograph by clicking onthe appropriate allograph box, then selecting the desired allograph forthe desired initial or final sound.

After the user has selected initial and final allographs, he or she mayinitiate a search for words matching the sounds associated with theseallographs by clicking on the Word Entry Box. One of ordinary skill intie art of programming would understand that one may use a separate orother indicator for this, such as a command button of a particularcolor, or with a symbol or word on it, or a key on the keyboard.

When the search for matching words is complete, a matrix or list ofselections is presented within display area 13, as shown in FIG. 5. Inthe exemplary Prototype, each selection in the matrix has a smallpicture of an animal associated with it, and the name of the animaldisplayed adjacent to the picture. When the user rolls the mouse over aselection, the name of the animal is spoken. When the user clicks on theselection, a display is shown within display area 13 containing: 1. Adefinition (description) of the animal, 2. An interesting fact about theanimal, and 3. The same picture of the animal. The definition andinteresting fact will each have an icon next to it which, if clicked on,will cause a .WAV file to be played speaking the displayed words. If theanimal itself is clicked on, or an icon adjacent to the animal, such asa filmstrip icon for example, a MCIWNDX control will appear with thefirst frame of the associated animation. The user may then start theanimation, and manipulate it, using a slider to go slower, faster, andbackwards. The user may view the animation as many times as he or shewishes.

One of ordinary skill in the art of programming would understand that ifonly one choice is matched with the initial/final phonemes, the list ormatrix need not necessarily be displayed, but only the display of theselected choice.

When the user wishes to select another animal, he or she clicks on theinitial or final sound input box. This causes the screen to appear inits initial configuration. One of ordinary skill in the art ofprogramming would understand that there are numerous ways to return tothe initial state, such as a commnand button, a keyboard key, and soforth.

FIG. 5 depicts an exemplary interface of the Phonic Engine after amatched list has been presented. In this figure, only an initial soundhas been used. The figure depicts the display generated by the exemplaryPrototype. With the sound /k/ having been chosen as the initial sound,pictures of a camel, cat, cow and kangaroo may be displayed, amongothers.

If this exemplary interface were to be adapted for use in conjunctionwith another type of system, for example, a "Pay Per View" interactivetelevision (ITV) system, the movie name "Casper", a picture of its maincharacter or a reduced size version of its print advertisement may bedisplayed among others. If the interface were used in an ITV, onlineservices, kiosk or other scenario where a child may have access tomotion videos showing the history, mechanics, science and otherinformation about certain inventions, pictures of a camera and a cuckooclock, among others, may be displayed.

One of ordinary skill in the art of multimedia programming wouldunderstand that the animation may be presented adjacent to thedefinition and interesting fact, or as a larger animation, temporarilyobscuring other information on the screen. In the latter case, one ofordinary skill in the art of programming would understand that means forreviewing the written material could be accomplished via clicking theword box, a command button, the original selection with picture of theanimal, minimized and placed appropriately on the screen, or anotherindicator appropriately placed on the screen or by a keyboard key.

In the exemplary Prototype, the searching process may be accomplished byprocessing an array of a user defined data type, where each array membermay include information used to select the array member and informationused to display the associated data, such as: indicators of initial andfinal (or just an initial) phonemes associated with the name of theanimal, a filename of a small picture for the list or matrix anddisplay, the name of a file containing associated animation data; thename of a file containing associated word, definition, interesting fact,etc. Members matching the initial and final (or just initial) phonemesare displayed in the above stated list or matrix. If the list or matrixoverflows the allocated screen area, scrolling or paging will be used todisplay the remaining selections. One of ordinary skill in programmingwould understand that the array may be an array of a user defined type,or another set of properly organized data structures; and that suchfeatures as filenames, file formats, data organization, and dataaggregation, for example, may be implemented in a variety of ways.

Another contemplated variation uses only the initial phoneme forselection. For a young audience, and for a database in which the numberof items is sufficiently limited that a set of a readily browsablechoices are likely to be displayed for any initial phoneme, this type ofselection process may be useful.

In another variation, each entry in a collection of meanings has noassociated text. Just sensory data about the meaning. For example, inthe Talking Dictionary or Prototype systems, it is not necessary to haveany text associated with an entry to locate it by orthographic orphonetic/phonemic component(s).

In another variation, the Phonic Engine is used in conjunction with atelevision receiver containing an electronic program guide (EPG) such asthe StarSight® system, available on Zenith® television receivers, whereusers can utilize it as a navigational tool to select programs. In yetanother variation, the Phonic Engine may be used in connection withinterfaces to interactive television systems or online services. Inthese applications, one may use the Phonic Engine to select merchandise,or perform other transactions which would otherwise not be selectable tothose with limited language capability. For example, if a child wants toview the program Sonic The Hedgehog®, or play the video game SonicSpinball®; and if the number of programs and/or video games was verylarge, the user could input indicators for the phonemes /s/ and /k/, or/s/ and /g/ (for Sonic, and Sonic the Hedgehog, respectively), be showna matrix of pictures of all corresponding items, of which the abovewould be a part; be able to browse the selection and (optionally) hearthe names of the selections pronounced, and finally select the desireditem. One of ordinary skill in the art of designing user interfaceswould recognize that the phonetic/phonemic and other indicators could beon-screen, on a simple remote device, or on a remote device with displayand selection capabilities.

In another variation, it would be apparent to one with ordinary skill inprogramming for networks, that this technology can easily be implementedacross networks, via TV or interactive TV or, online services, for bothnavigation and selection of games, programs, merchandise or othermaterials or information; in kiosks at children's museums and otherplaces where a child, or one with limited language capabilities, istrying to find information.

An on-screen user interface of this type may be substantially the sameas that shown in FIG. 1 except that it shows small pictures adjacent tothe words or may not show words at all. Alternatively, a display of thetype described above, with reference to FIG. 5 may be used.

In another variation, it would be apparent to one of ordinary skill inthe art of programming that the Phonic Engine could be incorporated intoa computer or other command interpreter. For example, an interfacesimilar to FIG. 1 would be displayed on a CRT screen, and a child or onewith limited language could use the interface as in the exemplaryTalking Dictionary. In this case one could display a list of commandscorresponding to computer commands, and the commands would besequentially pronounced, or pronounced when clicked on.

In another variation, blends and other orthographic information (e.g.prefixes or suffixes) may be used in conjunction with sensory elements(e.g. pictures or sounds) to lead to selection of a meaning. In thisvariation, the blends could be selected in the same fashion asallographs in FIG. 1, that is, by first selecting the letter with whichthe blend or other orthographic element begins.

The Phonic Engine may generally be applied as a software object to anytype of selection process in an application where a user provides a listof selections and corresponding language-based data, such as initial andfinal phonemes, prefixes, suffixes or other language components. In thisform, the Phonic Engine could be integrated with the application.

From a functional point of view, the visual interface presented by thisobject may be substantially the same as that shown in FIG. 1. Theobject, however, may allow a user to design an alternative interface byproviding a set of entry points and event handlers that could beassociated with controls or objects or a screen or other objects usingwell-known techniques. The interface may be adapted to provide onlyphonetic/phonemic or orthographic choices associated with the possibleselections and to yield user input in terms of, for example, the indexof a desired selection or the initial and final sounds of the desiredselection. Under appropriate circumstances, it may be desirable for theobject to accept only initial sounds or to accept language componentsother than initial and final sounds, such as medial sounds, prefixes orsuffixes.

The input data may be used by the object as search criteria for a listof selections provided by the application programmer. The resultingmatches may be provided for the user to browse, either visually, usingpictures provided by the programmer; aurally, using 1) sound-clipsprovided by the programmer, or 2) an internal text-to-speech translatorto say text provided by the programmer.

The visual interface presented by this object may be substantially thesame as shown in FIG. 1. The object may also allow a user to design analternative interface through a number of means, such as varying thesize, design or selection method of the controls; or by varying the typeof information (phonetic/phonemic, orthographic, etc.) provided to suitthe audience and/or the associated possible selections.

Integration may occur to different degrees. For example, the PhonicEngine could obtain the user input only, and pass it to a routinesupplied by an application developer; or could be aware of the databaseand selection criteria, search the database, and provide indexes orpointers of matched entries to a routine which would manage furtherselections/transactions; or it could be made aware of the nature of themedia within the database, receive further selection from the list ofmatches, and pass this to a supplied routine for further processing; andso on.

MECHANICAL EMBODIMENT

As mentioned above, mechanical devices may also be used to practice theinvention. An exemplary alternative mechanical embodiment may include aselectively punched card for each respective word in the dictionary ordatabase. U.S. Pat. Nos. 2,556,232 to Stiller, 2,647,519 to R. Brinkmannto Broxten and 4,081,079 to Roth et al. are hereby expresslyincorporated by reference for their teachings on the construction anduse of mechanical selection devices for card indexes.

The exemplary mechanical embodiment includes a device and technique suchas that described in U.S. Pat. No. 2,647,519 to R. Brinkmann.Familiarity with that patent is assumed, and a detailed description ofthe complete Brinlmann device is not repeated herein. In Brinkmann, afirst selector card is placed in tie front of a stack of index cards,and an identical selector card is placed in the back of the stack. Cardsare selected by placing a selector rod through a hole in the frontselector card, through the stack, and out through the rear selectorcard. The cards in the stack are allowed to drop, except for the cardshaving circular holes that line up with the holes in the selector cardsthrough which the selector rod has been inserted.

Reference is now made to FIGS. 4A and 4B. In the exemplary mechanicalembodiment of this invention, a plurality of word cards (not shown) arestacked horizontally, from front to back. Each word card has a firstcircular hole in a respective position corresponding to the first soundof the word, a second circular hole in a respective positioncorresponding to the final sound of the word, and a plurality ofelongated slots in positions that correspond to the remaining sounds ofthe language. The use of elongated slots and circular holes is explainedin detail in the Brinkmann patent incorporated by reference herein. Eachcard also includes a picture and spelling of the word. Additionalinformation may also be included.

Selector cards 302 and 322 (shown in FIGS. 4A and 4B) are provided forplacement at the front and back of the stack. To allow independentselection of initial and final sounds, respective selector rods are usedfor the initial sound and the final sound. Two sets of initial soundselector cards 302 are provided for initial allograph selection, and twosets of final sound selector cards are provided for final allographselection. This allows placement of selector cards both in the front ofthe stack and in the back, for proper alignment of the selector rods.

Initial selector card 302 (FIG. 4A) includes a picture 312 of a wordthat includes a respective allograph in the initial position, thespelling cf that word (with the pertinent sound highlighted), and asingle circular hole 313 corresponding to that allograph. Final selectorcard 322 (FIG. 4B) includes a picture 332 of a word that includes arespective allograph in the final position, the spelling of that word(with the pertinent sound highlighted), and a single circular hole 333corresponding to that allograph. Card 302 has an opening 314, and card322 has an opening 334. By placing an initial sound selector card 302and a final sound selector card 322 on top of one another, a pair ofcircular holes 313 and 333 is formed, corresponding to a respectiveinitial/final sound pair.

Once the user chooses an initial/final sound selector card pair 302,322, the chosen pair of selector cards is placed in the front of thestack of word cards. A duplicate selector card pair is placed in theback of the word card stack for proper alignment of the selector rods.One selector rod is inserted through the circular hole 313. Thenon-selected cards are then allowed to drop, as described in Brinkmann.All of the cards remaining on the selector rods at this point have thedesired initial sound. A selector rod is then inserted through thecircular hole 333, and the rod 313 is removed. The non-selected cardsare then allowed to drop. The remaining cards have both the desiredinitial sound and the desired final sound.

The user can peruse the remaining cards to find a card having a pictureand spelling of the desired word.

Other mechanical embodiments are also contemplated.

The potential benefits of the Talking Dictionary and the underlyingtechnology are numerous. It may increase literacy. Users may learn toexercise increased control and power over their environments through theuse of written language. The Talking Dictionary allows children to playwith written language. It provides children with a friendly, enjoyableenvironment which they can explore independently, motivated by their owninnate enchantment, all the while increasing their literacy. It mayfurther increase auditory discrimination and pronunciation skills.

While the invention has been described with reference to exemplaryembodiments, it is not limited thereto. Rather, the appended claimsshould be construed to include other variants and embodiments of theinvention which may be made by those skilled in the art withoutdeparting from the true spirit and scope of the present invention.

What is claimed:
 1. A method for specifying positional phonemicinformation with respect to a word or group of words, comprising thesteps of:providing a plurality of phonemic indicators, the indicatorssubstantially corresponding to phonemes; allowing selection of theindicators; receiving selection of an indicator; determining theselected phoneme and its corresponding position with respect to one ormore words.
 2. A method according claim 1, further comprising the stepof associating a position indication with at least one of the phonemicindicators.
 3. A method according to claim 1, wherein the positionalinformation comprises one of an initial position in a word and a finalposition in a word.
 4. A method according to claim 3, wherein thephonemic indicators are provided in a manner such that phonemicinformation represented by the phonemic indicators is understandable toan individual who cannot read.
 5. A method according to claim 4, whereinthe phonemic indicators are further provided with respectiveassociations between the phonemic information and positional informationwith respect to a word or group of words.
 6. A method according to claim5, further including the step of allowing a user to pre-select theassociations provided with the phonemic indicators.
 7. A methodaccording to claim 4, wherein the receiving step forms an associationbetween the phonemic and positional information.
 8. Apparatus forspecifying positional phonemic information with respect to a word orgroup of words, comprising:means for providing a plurality of phonemicindicators, the indicators substantially corresponding to phonemes;means for allowing selection of the indicators; means for receivingselection of an indicator; means for determining the selected phonemeand its corresponding position with respect to one or more words. 9.Apparatus according claim 8, further comprising means for associating aposition indication with at least one phonemic indicator.
 10. Apparatusaccording to claim 8, wherein the positional information comprises oneof an initial position in a word and a final position in a word. 11.Apparatus according to claim 10, wherein the phoemic indicators areunderstandable to an individual who cannot read.
 12. Apparatus accordingto claim 11, wherein the phonemic indicators further include respectiveassociations between the phonemes and their positions with respect to aword or group of words.
 13. Apparatus according to claim 12, furthercomprising means for pre-selecting the associations indicated by thephonemic indicators.
 14. Apparatus according to claim 11, wherein themeans for receiving includes means for forming associations between thephonemic and positional information.
 15. Apparatus according to claim11, wherein the means for receiving includes electronic equipmentcapable of conveying input signals, selected from the group consistingof computer selection devices, remote control devices, networks, andcombinations thereof.
 16. Apparatus according to claim 11, wherein theword or group of words represent information selected from a groupconsisting of dictionary entries, encyclopedia entries, referencematerial, educational topics, database entries, games, merchandise,television programs, and combinations thereof.
 17. Apparatus forspecifying positional phonemic information with respect to a word orgroup of words, comprising;a computer display providing a plurality ofphonemic indicators, the indicators substantially corresponding tophonemes; computer input apparatus which allows selection of at leastone of the indicators; a computer processor which receives the selectionof an indicator and which determines the selected phoneme and itscorresponding position with respect to one or more words.
 18. Apparatusaccording claim 17, wherein the computer display further associates aposition indication with at least one of the phonemic indicators. 19.Apparatus according to claim 17, wherein the positional informationcomprises one of an initial position in a word and a final position in aword.
 20. Apparatus according to claim 19, wherein the displayedphonemic indicators are understandable to an individual who cannot read.21. Apparatus according to claim 20, wherein the displayed phonemicindicators further include respective associations between the phonemesand their positions with respect to a word or group of words. 22.Apparatus according to claim 21, wherein the associations between thephonemes and their positions with respect to the word or group of wordsare pre-selected by the user.
 23. Apparatus according to claim 20,wherein the computer processor forms an association between phonemic andpositional information when it receives the indicator.
 24. Apparatusaccording to claim 20, wherein the computer input apparatus is selectedfrom the group consisting of a computer selection device, a remotecontrol device, a network and combinations thereof.
 25. Apparatusaccording to claim 20, wherein the word or group of words representinformation selected from a group consisting of dictionary entries,encyclopedia entries, reference material, educational topics, databaseentries, games, merchandise, television programs, and combinationsthereof.